Stability and approximation of invariant measures of Markov chains in random environments
نویسندگان
چکیده
We consider finite-state Markov chains driven by stationary ergodic invertible processes representing random environments. Our main result is that the invariant measures of Markov chains in random environments (MCREs) are stable under a wide variety of perturbations. We prove stability in the sense of convergence in probability of the invariant measure of the perturbed MCRE to the original invariant measure. Our approach makes no assumptions on the transition matrix functions representing the Markov chains except measurability with respect to the random environment. We also develop a new numerical scheme to construct rigorous approximations of the invariant measures, which converge in probability as the resolution of the scheme increases. This numerical approach is illustrated with an example of a random walk in a random environment. MSC 2010 subject classifications: Primary 60J; secondary 37H.
منابع مشابه
Stability and Approximation of Random Invariant Measures of Markov Chains in Random Environments
We consider finite-state Markov chains driven by a P-stationary ergodic invertible process σ : Ω → Ω, representing a random environment. For a given initial condition ω ∈ Ω, the driven Markov chain evolves according to A(ω)A(σω) · · ·A(σn−1), where A : Ω → Md is a measurable d × d stochastic matrix-valued function. The driven Markov chain possesses P-a.e. a measurable family of probability vect...
متن کاملOn the computation of invariant measures in random dynamical systems
Invariant measures of dynamical systems generated e. g. by difference equations can be computed by discretizing the originally continuum state space, and replacing the action of the generator by the transition mechanism of a Markov chain. In fact they are approximated by stationary vectors of these Markov chains. Here we extend this well known approximation result and the underlying algorithm t...
متن کاملADK Entropy and ADK Entropy Rate in Irreducible- Aperiodic Markov Chain and Gaussian Processes
In this paper, the two parameter ADK entropy, as a generalized of Re'nyi entropy, is considered and some properties of it, are investigated. We will see that the ADK entropy for continuous random variables is invariant under a location and is not invariant under a scale transformation of the random variable. Furthermore, the joint ADK entropy, conditional ADK entropy, and chain rule of this ent...
متن کاملContinuity of Generalized Semi-Markov Processes
It is shown that sequences of generalized semi-Markov processes converge in the sense of weak convergence of random functions if associated sequences of defining elements (initial distributions, transition functions and clock time distributions) converge. This continuity or stability is used to obtain information about invariant probability measures. It is shown that there exists an invariant p...
متن کاملEstimation of the Entropy Rate of ErgodicMarkov Chains
In this paper an approximation for entropy rate of an ergodic Markov chain via sample path simulation is calculated. Although there is an explicit form of the entropy rate here, the exact computational method is laborious to apply. It is demonstrated that the estimated entropy rate of Markov chain via sample path not only converges to the correct entropy rate but also does it exponential...
متن کامل